How to Choose the Number of Clusters: The Cramer Multiplicity Solution

Most of the data analysis of microarray gene expression data use a clustering algorithm, as a preprocessing step, in genomic functional analysis for example [6] or as the main discriminating tool, in the tumor classification study [9]. While from the experimenter point of view the simplest clustering method could be the best, it is acknowledged [8] that the reliability of allocation the units to a cluster and primary, the number of clusters are questions waiting for a joint theoretical and practical validation. Model based clustering methods [10] as well as machine learning methods [3, 2] lack from an a priori technique to determine the number of clusters. Usually the number of clusters is fixed such that some reliability criteria is maximized, as an a posteriori procedure, depending strongly on the clustering method. The work we did on clustering algorithms revealed an objective method of choosing the number of clusters, inspired by spectral learning algorithms and Cramer multiplicity. The clustering problem is mapped on the framework of spectral graph theory by means of the min-cut problem. That induces the passage from the discrete domain to the continuous one, by the definition of the eigenfunctions associated with the Laplacian operator of the graph. It is the analysis on the continuous domain allowing the screening of the Cramer multiplicity, otherwise set to 1 for the discrete case. An algorithm for the approximation of the number of clusters is implemented, by the use of the Legendre polynomials. The test set considered is the yeast cell data [4].